Emergent Antiferromagnetism in D-wave Superconductor with Strong Paramagnetic 

Pair-Breaking 



Yuhki Hatakeyama and Ryusuke Ikeda 
Department of Physics, Kyoto University, Kyoto 606-8502, Japan 
(Dated: January 14, 2013) 

It is theoretically shown that, in the four-fold symmetric d-wave superconducting phase, a param- 
agnetic pair-breaking (PPB) enhanced sufficiently by increasing the applied magnetic field induces 
not only the Fulde-Ferrell-Larkin-Ovchinnikov (FFLO) superconducting state but also an incom- 
mensurate antiferromagnetic ( AFM) order with Q-vector parallel to a gap node. This AFM ordering 
£NJ ■ tends to occur only below H C 2 at low temperatures, i.e., in the presence of a nonvanishing supercon- 

ducting energy gap A rather than in the normal phase. Through a detailed study on the resulting 
AFM order and its interplay with the FFLO spatial modulation of A, it is argued that the strange 
high field and low temperature (HFLT) superconducting phase of CeCoIns is a coexisting phase of 
the FFLO and incommensurate AFM orders, and that this PPB mechanism of an AFM ordering 
is also the origin of the AFM quantum critical fluctuation which has occurred close to -ff C 2(0) in 
several unconventional superconductors including CeCoIns. 
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O ■ I. INTRODUCTION 
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Recently, the presence of an antiferromagnetic (AFM) quantum critical behavior near the superconducting (SC) 
depairing field (or, the mean field upper critical field) H C 2(0) at low temperatures has been commonly found in 
several unconventional superconductors such as CeCoIns 0, Q, pressured CeRhlns 3], NpPd2Als Ce2PdIng 
and Tl-compounds of cuprates @ . Most of these materials belong to the so-called heavy- fermion superconductors and 
hence, are expected to have a large Zeeman term, i.e., remarkable Pauli paramagnetic pair breaking (PPB)^ effects. 
Conventionally, an AFM order is expected to be suppressed by the presence of a finite SC energy gap |A| 0, Q below 
the mean field SC phase transition, indicating a possibility of enhancement of AFM fluctuation or order above H c2 (0). 
However, a closer examination of the AFM critical behavior suggests the presence of an AFM quantum critical point 
(QCP) below H c2 (0) Q. In fact, measurements in the SC state of the heavy fermion superconductor CeCoIns showing 
a remarkably large PPB [§-ll[ in finite magnetic fields {H ^ 0) clearly show the presence of AFM fluctuation in the 
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SC state which is enhanced with increasing H up to H C 2 [12H14I ]. A schematic picture on the AFM critical fluctuation 
k*" ■ near H C 2(0) is represented in Fig.l. 

The incommensurate AFM order discovered recently through neutron scattering measurements IH 1(| at the high 



field corner of the H-T SC phase diagram of CeCoIns in H || ab will not be an independent event of the above- 
mentioned field- induced enhancement of incommensurate AFM fluctuation below H C 2. This AFM order has been 
detected just in the so-called high field low temperature (HFLT) phase of this material in H 1 c which has been 



previously identified 0, [TtJ with the long-sought spatially modulated Fulde-Ferrell-Larkin-Ovchinnikov (FFLO) SC 
state (HI. If focusing only on the magnetic properties seen through, e.g., the neutron scattering data [HI, [la], it 
might be natural to identify the HFLT phase with an incommensurate AFM phase coexisting with the spatially 
uniform d-wave SC order. However, the fact that the HFLT phase is destabilized by quite a small amount of not only 
magnetic impurities [3] but also nonmagnetic ones [13] is incompatible with the picture [la) favoring the presence of 
the uniform e?-wave SC order in the HFLT phase. In fact, it has been found 2l| that such an unexpected impurity 



effect is consistent only with the picture identifying the HFLT phase with a SC state with one-dimensional modulation 
parallel to H [l7j . such as a kind of the FFLO state. Then, it is necessary to clarify how this FFLO picture on the 
HFLT phase is compatible with the presence of the AFM order detected in the same phase. 

In the present work, we develop a theory comprehensively explaining the above-mentioned phenomena occurring 
at the high field side of the H-T SC phase diagram of a ci- wave superconductor with strong PPB such as CeCoIn 5 . 
To be specific, the favorable direction of the expected staggered moment is assumed throughout this paper to be 
perpendicular to the basal plane, i.e., parallel to the c-axis of the tetragonal structure of the quasi 2D SC materials 



according to an observation Ref.[15[ on CeCoIns. It is found by extending the conventional model on coexistence of 
SC and AFM orders to the nonzero field (H ^ 0) case with strong PPB that, in <i-wave superconductors with strong 
PPB, an incommensurate AFM order tends to be realized in higher fields but below H C 2(0) if the AFM Q-vector in 
the commensurate limit, called Qo hereafter, is parallel to the nodal direction of the SC energy gap. Throughout this 
paper, the component of Qo in the a-b plane of a tetragonal crystal is (tt, tt) so that the SC pairing state with a gap 
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FIG. 1: Field dependence of the AFM correlation length £af described schematically. For simplicity, the presence of a narrow 
HFLT phase just below H C 2 is neglected here (see sec.V and VI). The £af (solid) curve below H C 2 reflects the PPB-induced 
AFM ordering, while the solid curve above H C 2 results purely from the AFM fluctuation in the normal state. The dotted curve 
below H C 2 is the extrapolation of the high field curve to lower fields. If the i/ C 2-transition (the mean field SC transition) is of 
second order so that the SC fluctuation is not negligible above H C 2, the growth of £af upon decreasing the field in H > H C 2 
should, as indicated by the dashed curve, become sharper as a consequence of the coupling between the two orderings (see 
eqs.(29) and (31) below). 



node parallel to Qo is inevitably the d x 2_ y 2 -paired one. 

This PPB-induced AFM ordering or fluctuation has two crucial implications. First of all, the present AFM order 
occurs more easily in the SC phase with finite SC energy gap than in the normal state and thus, clarifies why no 
situation with AFM order only in the normal phase above H C 2(0) is seen in those materials [3-@| with an AFM critical 
behavior around H C 2(0). Second of all, since the presence or absence of the field- induced AFM ordering or fluctuation 
strongly depends on the relative orientation between the gap node direction and Qo, the present theory is also useful 
for addressing the pairing symmetry of those SC materials including CeCoIns : In the case with Qo parallel to (7T, 
7r), the presence of the field-induced AFM fluctuation near H C 2(0) implies the d x n_ y i -pairing symmetry of the SC 
material. 

The present field-induced AFM ordering is a characteristic property of superconductors with strong PPB together 
with the FFLO SC state and the first order H C 2 -transition [171, |22j, and hence, a consistent emergence of both this 
AFM order and the FFLO SC order in the same material is naturally expected. In fact, a couple of observations 
m the HFLT phase of CeCoIn 5 have been done so far which support identification between the HFLT 
phase of this material and a FFLO state with a modulation parallel [H [H[ to the applied field H, which will be 
called a longitudinal FFLO state hereafter 2jJ. For this reason, we also examine the field-induced AFM ordering 



in the longitudinal FFLO state by invoking the Pauli limit, in which the orbital pair-breaking inducing the vortices 
is neglected, to evaluate thermodynamics and construct a H-T phase diagram. It is found that the FFLO spatial 
modulation significantly extends the AFM ordered region so that the situation in which the AFM order is absent 
outside the FFLO state in the phase diagram is easily realized. The obtained results on the phase diagram will be 
compared with recent NMR data [28[ presenting a firm evidence of both the longitudinal FFLO structure and the 
AFM order in the HFLT phase of CeCoIn.5 in H _L c. A preliminary report of the present work can be found in 
Refs.[29[ and [301 ] . We note that the presence of PPB-induced AFM fluctuation has been first noticed through our 
numerical studies of effects of an AFM quantum critical fluctuation on the vortex form factor 29|, 31 1 . 



When we discuss the phase diagram including a field-induced AFM ordering in the text, a situation with an 
indication of the presence of an AFM QCP H* will not be distinguished from the case with a finite AFM transition 
temperature in some field range. In fact, the field at which the AFM transition temperature is the highest in the 
latter situation would correspond to H* if the former situation is realized, and the former is easily realized from the 
latter, i.e., by reducing the strength of the electron repulsion leading to the AFM ordering in the normal state. 

This paper is organized as follows. In sec. II, two theoretical methods for studying the AFM ordering in the d-wave 
superconductors are explained in details. In sec. Ill and IV, numerical results on the AFM ordering following from 
them are presented, and their implications are explained by assuming the d-wave SC phase not to include a FFLO 
spatial modulation. The corresponding results in a FFLO phase are considered in sec.V. Section VI is devoted to a 
summary of the present work and final remarks. 
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II. MODEL 



In the present study, we have examined a possible AFM ordering according to two approaches to be explained below. 
In zero magnetic field (H = 0), our starting electronic model is the same in the two approaches and can be expressed 
by the Hamiltonian including just the two interaction channels of a d-wave superconductivity and antiferromagnetism 
H = "Hkin + "Hafm + Use, where 

H ki „ = d]T fd 2 v ± [[^'(r^ltext-iVi)^^) - ^[^(rJlV^K) +h.c 

H AF = -t/^4(q)^(q): 



q,n 

^SC = -^l]l] W k4, CT C -k+q,-<rX) U;k ' C - k '+ { l- ff ' Ck '^' W 
q k.cr k'.cr' 

with 

<Sn(q) = C l,a( a ' fi )c/J c k+Qo+q,/9- (2) 

k,a,P 

In the above expressions, a (= ±1) denotes the spin projection, and the gap function satisfies the property u>k+Q = 
-Wk peculiar to the ^2 -pairing state. Further, in the case of a Fermi surface with perfect nesting, the relation 
e(k + Qo) = — s(k) is satisfied. Through our explanation of our theoretical expressions, the unit h = c = ks = 1 will 
be used. 

In the presence of a uniform magnetic field H, the spin quantization axis parallel to H will be chosen hereafter. 
Further, the dispersion relation e(— iVj_) needs to be replaced by s(— iVj_ + eA) + la, where / is the Zeeman energy 
and is usually written as [i^gH with a (/-factor and the Bohr magneton /xb- 

For the moment, we focus on the mean field approximation neglecting both the SC and AFM fluctuations, and 
roles of the AFM fluctuation will be discussed in sec. IV and V. The AFM staggered field m playing the role of the 
AFM, or spin density wave, order parameter is given by m(q) = U (Sfi(q)), where ( ) denotes the statistical average. 
When the AFM staggered moment carries a finite momentum q, Q = Qo + q expresses the incommensurate AFM 
modulation wave vector. Expecting a finite q to be uniquely chosen at the microscopic level, the mean field expression 
of "Haf may be represented by 

Haf.mf = t/-Vft(q)| 2 - [m fi (-q)5 fi (q)+h.c.]. (3) 

Here, the expression was written for general n. In the ensuing analysis, we assume n to be fixed to the c-axis of 
the tetragonal structure for any direction of the magnetic field based on an experimental report on this issue (TBj on 
CeCoIns. Thus, we will not write the index n in m„ hereafter. Implication of this assumption on the fixed n will be 
commented on in sec. III. Similarly, the corresponding mean field expression of Use is 



Hsc.mf = Isr'IAtO)! 2 



A *( ) X! -y CTC -k ,-cr C k ,cr + h.C. 



k.rr 

Here, the SC order parameter satisfying the gap equation 



(4) 



A = -y ^ WkO"(c_k,- CT Ck l£r ) (5) 

k,cr 

was, for convenience of description, assumed to be spatially uniform. We perform the mean field analysis of % in 
the following two approaches separately. One is a perturbative approach based on the microscopic derivation of a 
Ginzburg-Landau (GL) free energy of a form expanded in powers of both the SC and AFM order parameters. This 
method is useful in discussing, at least, the case with a relatively weaker PPB in which the i/ C 2-transition remains 
second order, because the orbital pair-breaking effect of the magnetic field inducing the vortices, necessary for obtaining 
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the second order _ff C 2-transition, is included in the familiar manner. We note that the second order iJ C 2-transition is 
not a consequence of the GL expansion in the SC order parameter, as have been demonstrated through a derivation 
of the first order iJ c2 -transition [13]. However, this method is insufficient for examining the detailed structure of 
the AFM order reflecting the quasiparticle's dispersion relation. Another approach is to focus on the Pauli limit in 
which the field-induced vortices are completely neglected. The neglect of the vortices is inappropriate for considering 
response properties in the SC phase such as the vortex elasticity, while it may not affect evaluation of thermodynamic 
quantities of superconductors with strong PPB significantly. Rather, the details of the AFM order can be examined 
numerically within this approach. Note that these two approaches are complementary with each other. 



Perturbative Approach 

First, let us start with explaining the perturbative approach. The quasiparticle energy incorporated in Hkin is 
assumed in this approach to satisfy 

£ (k + Q ) = -£(k)-T c0( 5i C , (6) 

where the dimensionless parameter <5ic measures the incommensurability, i.e., the deviation from the perfect nesting 
condition for a spin-density- wave or AFM ordering, Qo = (n / a : n / a,n / d) with the lattice constants a (in the ab 
direction) and d (in the c-direction) is the commensurate AFM modulation wavevector. Note that, here, 5ic is 
assumed to be a constant. The general case in which Sic is k-dependent will be studied in the Pauli limit approach to 
be given later. Further, the opened Fermi surface in the c-direction, i.e., the quasi 2D nature, might become important 
in some electronic processes, while it is safely negligible in considering spatial variations of the order parameter fields 
A in a vortex state and of m as far as the spatial anisotropy of the SC material to be defined later is relatively small. 
For this reason, the order parameter fields will be assumed 2J] to be functions of a continuous spatial coordinate 
r = (rj_, z) by replacing jd by z. 

Regarding the Green's function (t; rj_, r' ± ; j, j') = — (T T [tpj (rj_, r)(ipj, (r' ± , 0)) t ]) defined in the normal state, 
the quasi classical approximation for £/|^(rj_, r' ± ; j, j') = J^oIt (t; r±, r' ± )e t£ ™ T , i.e., 



Gg(r±,* f ±',3,j') - G£ ) (r±-r'±-J-3')\H=oexp(ie£ ± A(s) ■ ds 



(7) 



will be used, where e n — ir(2n + l)//3 is a Fermion Matsubara frequency, and j3 = 1/T is the inverse temperature. In 
diagrammatic calculations, the formula 

exp^-2iej^ A(s) • dsj A(ri) = exp (i(r x - r) • II) A(r) (8) 
with II = — iVj_ — 2eA(r) and 

gg>(k) = I d 2 r ± g£>(r x - r' x ; j - j')\ H=ae -^ ± -r' ±) ^k cd U-f) = _ _ _1_ _ _ (g) 

will be used. The orbital pair-breaking effect is incorporated through the relation (J7J. 

The contributions associated with the SC and AFM orderings to the free energy density are given by the sum 
/gl = /a + fff + /a + /"* + /fm + /im ■ The last sixth order term cx |A| 2 to 4 is not negligible in evaluating the 
sign of m 4 term in the free energy in the SC state. The purely SC contributions take the form 



/i 2) = <A*(r) 



\g\ {LL > 



A(r) 



Sp 



/i 4) = (< 4) (n t )A*(r 1 )A(r 2 )A*(r 3 )A(r 4 )| r ^ r ) , (10) 



where ( ) sp denotes the spatial average, and 

^i 2) (n) = ^ E Kl^k^t-k + n), 

Tl,k,(T 

k ( a\^) = ^ E Kl^ilH^^C-k + n^Ck + n^-n^^C-k + nt). (11) 
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On the other hand, fffl and fm^ are the corresponding GL terms in the AFM order parameter m under vanishing A, 
and, for simplicity, its q = case will be described here : 

f(2) = 
J m 

f(4) ^ 
J rn 



l - £ ^: ) (k)^)(k + Q )^)(k)^)(k + Q ) 



20 



n,k,a" 



Further, the coupling term between the two orders in the free energy density takes the form 



f(2,2) 
J Am 



2ir Am , 1 (IIi) + K Am<2 (Th) 



A*(n)A(r 2 )ro 2 | r 



where 



(12) 



(13) 



K Amil (iu) = -r 1 E kk| 2 ^: ) (k + Q )C ) ( k )^ ) (-k + n 2 )^(k-n 2 + nt), 

K Am , 2 {Ih) = J2 w k< +Qo ^: ) (k + n 2 )giX ) (-k)^ ) (-k + Q )^ ) (k + Q +nt). (14) 



In the above expressions, a is a in n || H and —a in n _L H, respectively. 

At this stage, one of key observations in the present work can be explained by noting that (k+Q ) = — (k) 
in the commensurate (<5ic — > 0) limit: In n || H case, it is noticed through comparison with ea. (|lll) that if Am, n 



(ri = 1 and 2) become the same expression as except the presence of lis' operations, and thus that, as well as 



the sign change of in higher fields resulting in the first order if c2 -transition [17|, |32j , they also become negative 



as PPB is enhanced. Since it implies that the m 2 -term in the free energy is reduced with increasing PPB, coexistence 
of the SC and AFM orders is favored, or an AFM QCP becomes closer with increasing H or by an enhancement of 
PPB. This conclusion is not limited to the d x i _ y 2 -pairing case and is satisfied for any pairing state. In contrast, the 
corresponding mechanism of a PPB-induced AFM ordering in the SC state in n _L H is peculiar to the d x 2 _ y i -pairing 
case and will be explained later. 

To perform the k-integrals, a particle-hole symmetry will be assumed to be approximately satisfied around the Fermi 
surface by introducing a constant density of states N(0) on the Fermi surface as a useful parameter. By replacing 
e(k + II) by e(k) + vt • II with the velocity Vk on the Fermi surface, we obtain 



Ai 2) (n) 



A(0) 



20 ^ 

71,(7 



ck(k) (| Wk i 2 ^: ) (k)^: ) (-k)) FS =7^-^(0) 



«sgn(e„)|w k | 



^ \ 2ie r 



21a - v k n 



= 2ntN(0) / dpf( P )( K| 2 exp 



-ip- 



vk n 



(15) 



where ( )ps denotes the angle average over the Fermi surface, f(p) = cos(2Ip/T c o)/smh(2irtp), t = l/(/3T c o), v(k) = 
<9e(k)/<9k, and the parameter integral 



dp e- Kp (Re k > 0) 



was used. As usual, the zero field SC transition temperature T c $ is defined by 



N(o)\g\ 



In 



0T ( 



E 

Sn>0 



2vr 

0£ n '' 



(16) 



(17) 



where e c is a high energy cut-off. 

In the presence of the orbital pair-breaking, the SC gap is varying in real space due to the presence of vortices. As 
far as the PPB is not extremely strong in the ballistic limit 26(, the vortex lattice solution may be described by the 
lowest Landau level mode of A. Then, we have 



A(r) = A^o(r). 



(18) 
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Here, (po is the familiar Abrikosov state 3 

<A)0) 



'- ) E ( 



i [ —v + -^s 2 ) - o ( — + sfc 



2 Vn? 



(19) 



expressed in terms of the integer s, which satisfies the normalization condition (|<A)| 2 )sp — 1< Further, rjj = l/-\/2|ei?| 
is the magnetic length for the Cooper pairs, and k = 7r 1 / 2 /3 1 / 4 for the triangular lattice. Noting that H± = ru(Ji x ± 
iH y )/y/2 is the raising and lowering operator for the Landau levels satisfying [II_,IT + ] = 1, we have exp(iT c "^ 1 Vk • 
Tip) = exp(-|r7| 2 /2)e"' n +e"'* n -, where r] = p(v x - iv v )/ {V2r H T cX) ). Using the property (^{r)e~ lpT -o lv ^ U ip Q (r)) sp = 
exp(— |ry| 2 /2), the quadratic term in the free energy is expressed in terms of eq. (|15[) by 



f (2) _ 
J A — 



N(0) 



In 



T c0 f3 



2irt / dp ( \wk 



sinh(27rfp) 



f(p) exp 



4r 2 T 2 ' 



FS. 



<|A| 2 ) SP 



(20) 



J x ' "y 



Deriving the corresponding expression of the SC quartic term is performed in a similar manner. The kernel in 
.pi takes the form 



i^ 4) (n 4 ) ~ nTNi ° ] E( [2i£n + 21a + v k • ni][2i £ „ 72ll + v k k ' n 2 ][2i £n + 21a + v k • II*] ) FS + ^ ° ^ 



27rfiV(0) 



r. 



^ / ft ^ /(pi + P2 + p 3 )(kk| 4 e lT - 1 ^ v - n i + ' ,2Vk - n2+MVk - n ^) F s + (n 2 o n 4 ). 

o Jo i=1 



By using the identity 

v k n 



exp ip 



we have 



/i 4) = 2ntN(0)T? 



fc 2 V 



E 



exp 



S— — OQ 



sk 7T , 

—y + tts' 

fH 2 



1 / x 



2 V r 



H 



sk + v2r] 



(21) 



(22) 



/>OQ 3 t 

/ n^^i+^+^^E^ 1 )' 1 ' 2 ^ 

JO - i V 27T , , 

1=1 (1,12 



-5(S+I2)* a 



w k exp 



where 



1 



1 



Re[e- po ] 



FS. 



|A| 



Pa = + ^2 2 + ??3 Z ) ~ jfe — r/x — Vlf - -/= [Jife - »?* + ^3) + fe^a + - J7|)], 

% = - iv y )/(V2r H T c0 ) in H || c, and II4 = II*! U 2 + II3. 

In H 1 c, the corresponding expressions to eas. (|20[) and (|23[) are given by replacing ^ and v y in H || c by 7 



(23) 



(24) 



and 7 1 / 2 w z , respectively, where 7 = ((w 2 )fs/(w 2 )fs) 1 ^ 2 = 2J5f(1 — J / 'E-p) 1 ^ 2 / '(it J) is the anisotropy of the SC length 
scales, and E-p is the Fermi energy in 2D (J — > 0) limit. 

(2) 

Next, the quadratic term f m will be rewritten by assuming the AFM staggered field to be uniform. A sign change 

f(2) 

density is positive (see below). By defining the Neel temperature In in the normal phase according to eq. (fl7|) with 

(2) 

\g\ and T c0 replaced by U and In, respectively, f m in n || c || H becomes 

1 



of fm' determines a position of the second order transition to the AFM phase if the 0(m 4 ) term in the free energy 

Neel 

f(2) 
m 

inf^i+i^Re 



I [ m ] = N(0) 



where ip(z) is the digamma function 



-7 + 



1 .Iaf3 McP 

^2 +l ^ +l ^r 



t(—-—\ 

n— N / 



(25) 



(26) 
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The Zeeman energy term plays a similar role to the incommensurability Sic for the AFM ordering in this field 
configuration. 

On the other hand, in n 1 H, fm becomes 



In 



1 



Re 



m 2 . 



(27) 



Within the present model in which no k-dependent anisotropy is assumed in the Zeeman energy term, the AFM 
transition temperature is _ff-independent even in the normal phase in this field configuration. 

(2 2) 

Now, let us turn to evaluating the kernels in the coupling term f\' ■ To be specific, in this subsection, we focus 
on the d x 2_ y 2 -pairing case in which w k = — u; k+ Q . The corresponding results in the d^-pairing case are given by 
changing the overall sign of K^m,2 in the following expressions. In the n || H \\ c case, we have 



K Am ,i(Tli) = -27r/3" 1 iV(0)^ 



zsgn(e„) \w k \ 



[2ie n + 21a + v k ■ U*i][2ie n + 21a + v k • U 2 ][2ie n + 21a + T c0 5 lc ] 



zsgn(£„)|w k | 2 



[2ie n + 21a + v k • n*][2ie„ + 21a + T c0 S lc ][2ie n + 21a + T c0 Si C + v k • (II* - n 2 )] 



FS 



2^7V(o)r- 2 / Y[dfH /(]>>)( Kl 

J ° i= l i= l \ 



e iSlc.p3 e iT c0 v k -(pin*+/> 2 n 2 ) 



J^ic(P2+Pa) e iT c J v w ((p t + p 3 )Til- p 3 Ti 2 ) , ^ c 



FS 



^A m ,2(n,) = -2ntN(0)T- 2 f fldpi /(Y"pi)(K 



i=l i=l 

i<5lc(P2-P3)pjr~ 1 V k -(p2ni^(pi+p2)n 2 ; 



= i<5lc(-P2+P3) p iT c0 1 V k -((pi+p3)ni-p3n 2 ) 



h.c. 



(28) 



Then, the coupling term of the free energy is expressed, using the relation 
(e^ 1 ^-0>i n t+P' n *)ip*(r 1 )<p (T2)\r i ^r)ap = e-^Xhi I'+l'? 2 l 2 + 2 >d* , i n the form 



/a™ = 27rW(0)T 2 / JJd Pl /(Pi+P2 + p 3 )(kk| 2 [4cos( ( 5 Ic p 3 ) e -^+'' 2 )^^^/( 2 '-« T -^ 



!sin((5icp2)sin((5icP3)e 



-p 2 1 (v±) 2 /(2r H T c0 ) 2 



FS. 



|A| 2 TO 2 

Ti2 712 
± cO ^ cO / sp 



(29) 



On the other hand, in the Hlc and n || c case, the corresponding expressions to eqs.(|28|) and (|29| are 

/>oo 3 



i^Am, l(n») 



2?rf 



iV(0) 

T c0 ^0 ^ smh[27rt( Pl + p 2 + p 3 )} 



w k 



cos( 2— (pi +p 2 ) )e 



'•<5icP3pjT , c0 1 v k -(pinj+p2n 2 ) 



cos( 2—p 2 ]e~* ' i 



AT(0) 



iftcCpi+pa)^^ v k -((pi+p 2 )nj-pin 2 ) 
2?rf 



h.c. 



PS 



r c0 ./o sinh[2^( Pl +p 2 + p 3 )] 



00s f 2^- (pi - p2 )) e -^P3 e »T- 1 v k .(p 1 ni-p 2 n 2 ) 



cos( 2— (pi-p 2 ) e r 

J c0 



*icP3 „iT , c0 1 v k -((pi+p3)ni-(p 2 +p3)n 2 ) 



+ h.c. 



(30) 



FS 
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and 



f (2,2) 

J Am 



N(0)T c 



f - - 

/„ n*. s 



2?rf 



sinh[27rf(pi + p 2 + P3)] 



4 cos 2/ 



T, 



)cos( ( 5 I cp3) e - (pl+p2)2 ^ /(2rffTco)2 



4 cos 21 co S (S lc ( Pl +p 3 ))e-P^/^ T ^ 2 



4 cos 21 



22- 

Pi - P2 



T, 



rl) 



cos(5i C p3)e 



-(pi-P2f(v±f/C2r H T c0 ) 2 



PS 



J c0 / sp J cO 



(31) 



The above K& m ^ n (n = 1 and 2) expressions clarify how the AFM ordering in the SC state innlH case occurs: 
For simplicity, let us consider again the commensurate case with vanishing Sic and neglect the orbital pair-breaking. 
Then, it is easily found that, in low T limit, KAm,i approaches the positive value N(0)/(2I 2 ), while — K\ m ^ 2 
shows the logarithmic divergence i" _2 iV(0)lii(7r _1 i"/T). Note that the term appearing through the anomalous Green's 
functions grows with a minus sign. Since this overall minus sign of m, 2 inducing an AFM ordering is a consequence 
of the property Wk+Q = -Wk in the d x 2_ y 2 pairing case, this tendency of an AFM ordering in the SC state is peculiar 
to a d-wave pairing symmetry with a gap node parallel to the expected Qo-vector of a commensurate AFM order. Of 
course, the divergence indicated above in low T limit is, strictly speaking, an artifact of the use of the GL expansion 
with respect to A. In section IV, however, it will be shown that the corresponding growth of the anomalous term of 
the coupled term of AFM and SC orders with a negative sign is satisfied beyond the GL expansion and thus that the 
PPB-induced AFM ordering innlH should occur generally in any d a .2_ y 2-wave paired superconductor with strong 
PPB under a high magnetic field. 

To examine the character of the AFM transition, the terms quartic in m have to be examined. In this perturbative 
approach, they consist of the normal contribution fffl and the additional term f^m^ including the SC contribution of 
0(| A| 2 ). As a broad tendency, the SC contribution f^m seems to make the AFM transition a continuous one even 
if, as is seen in H || c in some cases, fm is negative. The expressions of these two terms in the free energy density 
will be given in Appendix. 



Pauli limit 



In the charged systems leading to superconductivity at low temperatures, the two field-induced pair-breaking 
processes, the spin effect (i.e., PPB) and the orbital one inducing the vortices, need to be taken into account. In 
particular, when studying fluctuation effects and SC response properties such as the elastic responses peculiar to 
the vortex states, as pointed out elsewhere [lij], the use of the Pauli limit in which the orbital pair-breaking effect is 
neglected could lead to erroneous results on physical properties. On the other hand, most of thermodynamic properties 
and the high field phase diagram of bulk superconductors with strong PPB are, in the mean field approximation, 
reasonably described by the Pauli limit fll\, l32l|. Paying attention to this point, we present here a formulation in the 
Pauli limit of the FFLO states and the AFM ordering in superconductors with strong PPB where the i? C 2-transition 
is expected to be of first order at low temperatures. For simplicity, we assume here a 2D Fermi surface for which 
J = 0. 

The Matsubara Green's functions will be defined in the form 

G^(r;r ± ,r' x ) = -(T T [^\r, r)[^]\r', 0)]), 

F (ff) (r;r ± ,r' ± ) = -(T T [([^-^(r, r)[^(r', 0)]), 
FW(r;r ± ,rl) = -{T T [^\v, r)^ (r', 0)]), 

G (ff) (r;r ± ,rl) = -<T T [[^]t(r, r)^ (r', 0)]), (32) 

where the notation of the Green's functions has been changed to avoid a unnecessary confusion. Alternatively, they 
will be used often in the matrix form 



(33) 



Then, the Fourier-transform of G (a \ G^ } (k;R) = / dre le " T J d 3 (r ± - r'^G^ (t; r, r , )e- ik < r± - r '±\ where R 
(r + r')/2, satisfies 
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ie n -e(k)+Ia -A k (R)<7 
A k (R)<j -ie n - e(k) - la 



G^(k,R) = i + [v k ■ 3R]G^(k,R) 
where A k (R) = A(R)Kj k , and the derivative operator will be defined as 



(34) 



n = -iV R - 2eA(R) for A(R) 
9 R = \ nt = -iV R + 2eA(R) for A*(R) 
-iVr otherwise 



(35) 



Next, since a possibility of the FFLO state is considered, the Green's function will be expanded in powers of <9r in 
the way 



^ — "(O) ' "(2) ~T "(4) 1 ' 



(36) 



where G^l is the n-th order term of G in the gradient. The terms with odd n have been neglected above which do 
not contribute to the free energy density. Each term in the expansion ()36p is given by 



^l!(o)(k,R) = 



ie n — e(k) + la 



-A k (R) ff 



1 



where 



and 





} (2) (k,R) 


- U (0) 




!(4)( k > R ) 


- U (0) 



A* (R)cr -is n - e(k) - la 

D={e(k)} 2 -(ie n + Ia) 2 + \A k \ 2 



v k -5R( G^Jvk • d R G { °] 



is n - e(k) - la A k (R)a 
-A*(R)cr ie„ - e(k) + la 



(37) 



(38) 



v k • dn [ G^v k ■ d R [ G^v k • 5r [ G^ ; v k ■ d R G 



'(o) 



(<0, 



(o) 



r (0) 



r (o) 



(39) 



In writing down the expression of the free energy, the magnitude of the SC energy gap |A| has been assumed to 
be much more rigid compared with that of a possible AFM order parameter. This approximation is reasonable when 
the 7J C 2-transition is discontinuous. Then, |A| may be determined selfconsistently just from the SC part of the free 
energy density /a, which becomes [35j j 



/a = 



|A(R) 



- T 

2/3 ^ 



dcu Tr 



ia z G^ (k,R) 



(40) 



Sp 



As well as G, /a may also be classified in the form of gradient expansion [3£ 



AO) 

J A 



f (2) 
J A 



Ai) 

J A 



£„>0 k 

o?- 



( £ 2 +[e(k)] 2 + |A k | 2 -J 2 ) 2 +4 £ 2 / 2 

( £ 2 + [ £ (k)] 2 -/ 2 ) 2 +4 £ 2 / 2 

|v k • nA k | 2 

x e„>0 k 

2 (2[ £ (k)] 2 - ^ + I 2 | Ak| 2 )(af - 6a 2 & 2 + bj) 4a s 6 2 (a 2 - 6 2 
3 



Sp 



■(v k - V|A k 



2\2 



^EE 



£„>0 k 
12 , J2 , I A. |2 _ 7-2 



(a?+6 2 ) 4 

2 (2[ £ (k)] 2 - e 2 + 1 2 — \A k \ 2 )(aj - 6a 2 6 2 + bj) 4a s fr 2 (a 2 - b 2 ) 

3 (a 2 + 6 2 ) 4 



|(v k -n) 2 A k | 



(41) 



Sp 



where a s = [e(k)] 2 + e 2 + |A k | 2 - I 2 , and 6 S = 2e„7. 
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In our analysis in the Pauli limit, the only R-dependence of the SC order parameter we consider is that of the 
longitudinal FFLO state 



A(r) = 



V^COS^LO X) 



(42) 



in H || x, where just a single Fourier component with the wave length 27t/<7lo is assumed for the FFLO modulation, 
while the two kinds of d-wave pairing symmetries 



«'k 



cos(fc x a) — cos(fcj,a) (o? ;E 2_j ) 2-wave) 
sm(k x a) sm(k y a) (d xy -w&ve) 



(43) 



will be considered. Further, to examine the details of the resulting AFM order in the SC state, the dispersion relation 

e(k) = — 2ti(cos(k x a) + cos(k y a)) — 4t2 cos(k x a) cos(k y a) — 2t3(cos(2k x a) + cos(2k y a)) — fi (44) 

will be used following Ref . [36| . An incommensurability of AFM order primarily stems from a nonvanishing t<i term. 

A possible AFM order may be considered in the form of a Landau expansion in the staggered field m of the free 
energy density, in particular, if the AFM transition is of second order. Then, the AFM contributions in the free energy 
density take the form f m = fffl + fffl as a power series in m, where 



K2) _ 



{/-I _ x (n) _ x (an) 



InHln with n _L ao, 



(an) 



n.k.cr 



(45) 



(46) 



and 



(4) 



- E 

213 ^ 2 

e„,k,cr 



a z Gf n [ (0) (k + Qo)*,G£> (0 ,(k-)a z GfJ IQ) (k + Qo)& s G£ } . l0 ,(k) 



m 



It is found that these expressions are rewritten in the form 

4[4 + / 2 -£(k)e(k + Q )]aj 



X 



x (nj 
(an) 



^EE- 

£„>0 k 

-r 1 E E 



4A k A£ +Qo a ± 



a 2 + o 2 ' 

£„>0 k ± ' ± 



(47) 



(48) 



a ± = (4 + [e(k)] 2 + |A k | 2 -/ 2 )(4 + [ e (k + Q )] 2 + |A k+Qo | 2 -/ 2 )+44/ 2 , 
b ± = 2 £r J([ £ (k + Q )] 2 -[e(k)] 2 + |A k+Qo | 2 -|A k | 2 ), 



(49) 



and 



c(2) 



4a i 



£„>0 k 



6i 



7 2 - £ (k) £ (k + Q )-A k A k+Qo ; 



(50) 



£„>0 k 



(e 2 + I 2 - e(k) e(k + Q ) - A£ +Qo A k ) s 



ei [e(k) - e(k + Q )] 2 + / 2 |A k - A k+Qo | 2 - |e(k + Q )A k - e(k)A k+Qo | 



2a 1 b\ lm 4 . (51) 
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FIG. 2: Field dependence of the coupling term /a^ (solid black curve) of the free energy density in H _L c calculated in the 
perturbative approach and taken at t — 0.3. The used parameters are ap,± = 0.3, <5ic = 1.1, and 7 = 4.5. The upper (red) 
dashed curve is the contribution from if Am, 1, while the lower (blue) dotted one is that from if Am, 2. 



In n || H || c, 



where 



"II 
&11 



(an) 



4[(4 — P — e(k)e(k + Qo)a,| + 2e„J6ii] 



£„>0 k II II 

4A k A* +QoQ || 



£„>0 k 



+ |e(k)] 2 + I A k | 2 - 7 2 )(4 + [e(k + Q )] 2 + | A k+Qo | 2 - I 2 ) Ae 2 J 2 
2e n I(2e 2 n - 2I 2 + [e(k + Q )] 2 + [e(k)} 2 + |A k | 2 + |A k+Qo | 2 ) 



(52) 



(53) 



r(2) _ 



4[(eJ - / 2 - [e(k)][e(k + Q„)] - A k Aj; +qo )o, + 2ejb,] 



i>0 k 



6 2 



(54) 



/i 4) = EE 



2/3" 



e„>0 k 



(a| + 6 2 ) 2 

I A k - A k+Qo | 2 ) - | e (k + Qo)A k + e(k)A k+Qo | 2 
2(e 2 -I 2 - e (k)e(k + Q ) + A k+Qo A k ) 



(e 2 - / 2 - e(k)e(k + Q ) + A k+Qo A k ) 5 



7 2 )([ £ (k)+ £ (k + Qo)] 



4£„7a||6|| 



[ £ (k)+ £ (k + Q )] 2 + |A k -A k+Qo | 



(55) 



The expressions given in this section are used to examine the resulting H-T phase diagram and the details of the 
expected AFM ordering. 



III. CASE WITH SECOND ORDER /^-TRANSITION 

In this section, the H-T phase diagram near H C 2(0) and possible AFM ordering in the case with a moderately strong 
PPB will be numerically examined in terms of the theoretical expressions in the first half of the last section. In this 
case with a moderately strong PPB, the situation with first order //^-transition occurs rarely, and the H C 2 -transition 
remains of second order even in low T limit in most cases. This situation will be appropriate for explaining phenomena 
in the pressured CeRhIn5 0, Ce 2 PdIn 8 d, and other d x 2_ y2 -paired superconductors [6||. 
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FIG. 3: AFM transition curves described in H-T phase diagram in H i c as a function of Tn which is the Neel temperature 
in the normal phase in the commensurate limit. Here, calculation was performed consistently with that in Fig. 2. The AFM 
transition curves follow from the Tn/TcO values in the range between 0.575 (rightmost) and 0.16 (leftmost). The AFM order in 
the normal state (i.e., in H > H C 2(0)) is absent when Tn < 0.32T c o. The i/ C 2-transition on the dotted curve is of second order 
even at low T because the orbital pair-breaking at this moderate value of ap t ± is not negligible even in low T limit. 



The strength of PPB is measured by the dimensionless parameter ap = I[H = H c2 [T = 0)]/ (2ttT c o), where 1(H) 
is the Zeeman energy. The so-called Maki parameter olm corresponds to ap multiplied by the factor 7.1. Since we 
focus here on the family of quasi 2D materials, the following two PPB parameters will be defined here in the manner 
depending on the direction of H 

I[H = H^\\o)} 
aP '" - 2^ ' 



<*P,JL = TTTf 2 , (56) 

where and -f^jix = are ^ ne or bital depairing field in H || c and H 1 c, respectively. 

In this section, we show only calculation results obtained in terms of material parameters leading to an AFM order 
at finite temperatures. In systems with moderately strong PPB, no true AFM order has been detected so far, and 
just the presence of an AFM quantum critical fluctuation enhanced close to H C 2(0) have been found. But, the field 
at which the AFM transition temperature is the highest would be transmuted to an apparent AFM QCP through a 
slight tuning of material parameters or including an introduction of impurity disorder. 



H_Lc 



First, calculated results in the in-plane field configuration H 1 c will be explained. A tendency of the PPB-induced 

(2 2) 

AFM ordering is reflected in a field-induced sign change of the coupling or mixing term of the free energy 

(2 2) 

density. An example of this sign change of f&' m (H) is given in Fig. 2. In the figure, the field dependence to be seen in 
a conventional type II superconductor with a negligibly weak PPB is limited in low enough fields, H/ Hp < 0.2, where 
the field-induced reduction of /jf^ will not be able to be distinguished from the conventional picture @, Q on the 

(2 2) 

AFM ordering stemming from a reduction of f&' m due to the field-induced reduction of |A|. Namely, if the orbital 
depairing field lies in such a field range that the PPB is negligible for the disappearance of superconductivity, the 
field-induced AFM ordering would be regarded as being due to the vanishing of |A|. As is seen below, however, a close 
inspection of the behaviors near H C 2 indicates that this PPB-induced AFM ordering is weakened by the field-induced 
reduction of |A|. 

In Fig. 3, AFM transition curves in the field range around H C 2(0), obtained as a function of the Neel temperature 
7n in the normal state in H > H C 2, are shown. Here, the same set of material parameters as in Fig. 2 have been 
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FIG. 4: AFM transition curve described in H-T phase diagram in H || c and obtained in terms of 7 = 4.5, Op m = 0.39, 
Sic = 1.1, and Tn/TcO = 0.6. The _ff C 2-transition on the dotted curve is of second order. 

used, and the pressure dependence in real systems has been assumed to be directly reflected in that of Tn m the 
normal state. We expect Fig. 3 to be comparable with the corresponding experimental phase diagrams on CeRhliis 
(see Ref.Q and Figs. 4 and 20 (a) of Ref . ) . In the present electronic model, it is expected that not only the inverse 
of the electronic repulsive interaction, l/U, but also the incommensurability \5jc\ increase with increasing pressure. 
As mentioned previously [2^], an increase of JJjcl tends to enhance the PPB-induced AFM ordering (see also Fig. 5 in 
sec. IV). In the case of pressured CeRhlns 0, [37j], however, it will be natural to expect that the pressure dependence of 
the AFM phase boundary is determined by that of Tn in the normal state [38| , while that of the incommensurability 
is a correction. Hereafter, we focus on the behavior in the SC state of Fig. 3 that the AFM order disappears close 
to but below H C 2(0). A couple of remarkable features are seen in Fig. 3. First of all, at lower pressures (i.e., higher 
Tn), the AFM transition temperature monotonously decreases with decreasing field. This behavior indicates that the 
reduction of the SC energy gap |A| rather than the PPB-origin plays a dominant role for an increase of the AFM 
transition point, because the orbital pair-breaking is more dominant than PPB at such high temperatures. As Tn is 
sufficiently lowered, however, the PPB primarily determines the field dependence of the AFM transition temperature, 
and the AFM phase appears just below H C 2 rather than above H C 2. Note that, very close to H C 2, the AFM order is 
rather lost in the present case where ff C 2-transition is of second order. It implies that this AFM order is purely of SC 
origin and hence that it is lost as a consequence of the decrease of | A| on approaching H C 2 from below. This feature 
is one of the features clarifying that this AFM order is induced not by the decrease of |A| but rather by the PPB 
which is effective only in the SC state. 

H || c 

Next, the corresponding results on the PPB-induced AFM ordering in H || n and n || c will be briefly discussed. 
In contrast to the case in H _L n, the AFM ordering in the normal state in this case is suppressed, as well as the SC 
ordering, by the applied magnetic field in the present electronic model [38j |. Thus, the tendency of the PPB-induced 
AFM ordering is weaker compared with that in the last subsection. 

In this field configuration, both of the two coefficients i^Am,n (n = 1 and 2) in the coupling term change their 
sign with increasing the field and induce the PPB-induced AFM ordering. A typical phase diagram in this case is 
shown in Fig. 4. The main feature such that the AFM ordering tends to be promoted by PPB is qualitatively the 
same as in H 1 n. Reflecting the field-induced suppression of AFM order in the normal state mentioned above, 
however, the AFM ordering in this case is quantitatively weaker than that in H _L n, and the field region in which the 
AFM ordering is the most favorable tends to be shifted to lower fields than H C 2(Q). Our assumption in sec. I that, in 
CeCoIn 5 , n is locked to the c-axis irrespective of the direction of the field is closely related to this fact, because the 
AFM order just below H c2 (0) is not realized in CeCoIn 5 in H || c (39[. This issue will be discussed further in relation 
to Fig. 9 again. 
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FIG. 5: Upperlimits, defined by the vanishing of Xs, of possible AFM transition curves in H _L c obtained in the perturbative 
approach for Sic = 0.44 (lower curve) and 0.63 (higher one). For each case, an actual transition curve is lower than the dashed 
curve depending on the value of the repulsive interaction U. The parameter values, 7 = 4.5 and ap t ± = 1-1, are common 
between the two dashed curves. 




FIG. 6: Typical ~Xs v.s. H/Hp curve (solid black curve) taken at t = 0.1 in the strong PPB case in H 1 c. This figure 
should be compared with Fig.2 with a second order ff C 2-transition. The upper and lower dotted curves express the SC part of 



(n) and 



-x 



(an) 



, respectively. Calculation was performed in the Pauli limit fully taking account of the |A| dependences and 



assuming the two-dimensional circular Fermi surface and the incommensurability Sic = 0.1. 



IV. CASE WITH FIRST ORDER # c2 -TRANSITION AND AFM Q- VECTOR 

In this section, we explain how the results in the last section are changed when the PPB is much stronger so that 
the i? C 2-transition at lower temperatures is of first order. For brevity, the resulting SC state is assumed through this 
section to be spatially uniform at least in the direction parallel to H. Consequently, the only additional transition in 
the SC state in high fields is the PPB-induced AFM ordering. We stress here that, as explained in sec. I, the resulting 
coexistent phase of the AFM order and the uniform <i-wave SC one must not be identified with the HFLT phase of 
CeCoIns. The main purpose of this section is to clarify further details of the PPB-induced AFM ordering in addition 
to the obtained results in the last section. Relevance to the HFLT phase of CeCoIns will be discussed in details in 
the next section. 



H _L c 



In most part of this section, we focus on the H 1 c case. First, let us start with pointing out an important difference 
between the PPB-induced AFM order and the ordinary itinerant AFM one in the normal state. As demonstrated 
in Fig. 3, the AFM order usually diminishes with increasing pressure. However, applying the external pressure also 
enhances the incommensurability of the Fermi surface measured by \S\c\ or ^l- In relation to this, we present 
in Fig. 5 two lines implying the <5ic-dependence of the positions on which the SC part of the bare susceptibility 
Xs = X^ ~ X (A = 0) + X vanishes. This figure has been obtained by using a larger Maki parameter olm — 7 
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FIG. 7: Fermi surface (solid curve) following from the dispersion relation, eq. (|44p . with the values ii = 20T C , tiltt — —1.25, 
ts/ti — 0.65, and fi/ti — 1.85 and leading to a diag onal wave vector of the incommensurate AFM order (see Fig. 8 below) 
consistent with the experimental observation [IH liq j. The lower left dashed curve is the branch obtained by performing a 
Qo+ (0.37, 0.37) - shift for the original Fermi surface, while the dotted curve is the Fermi surface obtained [361 ] by mimicing 
the result from the band calculation. For the Qo— (0.37, 0.37)-shift, essentially the same nesting condition in the diagonal 
direction is obtained in the region — n < k x , k y < 0. Note that the magnitude of the density of states at each k = (k x , k y ) is 
represented by the colors. 



leading to the first order -ff C 2-transition at low temperatures compared with that in Fig. 3. Strictly speaking, a 
possibility of appearance of a FFLO state needs to be considered in the case of Fig. 5. The situation with a coexistence 
of the AFM and FFLO orders will be discussed separately in the next section. 

Note that Fig. 5 shows that the present PPB-induced AFM ordering is enhanced with increasing the incommen- 
surability. Since the normal part of the susceptibility, determining Tn used in Fig. 5 as the key parameter dependent 
on the pressure, is not incorporated in Fig. 5, and the pressure dependences of Tn and the incommensurability are 
competitive with each other for the PPB-induced AFM ordering, this hgure means that, in the case with a stronger 
PPB, it cannot be concluded generally whether the AFM ordering occurring close to H C 2(0) is enhanced or diminished 
with increasing pressure. 

The corresponding data to Fig. 2 in the last section are presented in Fig. 6, where, for brevity, the orbital pair-breaking 
effect (the presence of the vortices) has been neglected. Reflecting the discontinuous nature of the i? C 2-transition, 
Xs discontinuously vanishes at H C 2 with increasing field. In the region in which Xs > 0, the PPB-induced AFM 
ordering is possible depending on the value of the normal part of the susceptibility. This figure clearly shows that the 
PPB-induced AFM ordering is not an artifact of the logarithmic divergence of the anomalous part upon cooling in 
the perturbative approach. 

Next, as one aspect representing the resulting AFM order, the incommensurate part q of the AFM modulation 
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FIG. 8: H and T dependences of the diagonal q resulting from the Fermi surface (solid curve) in Fig. 7. This fact that q is 
nearly independent of H and T is consistent with the observation in Ref . [TBI, [ill . supporting the picture that the basic origin 
of making the incommensurate AFM wave vector in the HFLT phase of CeCoIns diagonal consists in the electronic structures 
and the details of the Fermi surface. The corresponding phase diagram is given later in Fig. 10 (b). 
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FIG. 9: Curves including a — \ s v.s. H/Hp curve (black solid curve) in H |j c which correspond to those in Fig. 6. For 
comparison, the corresponding — \ s (H) in d^y-pairing case (uppermost curve) is also shown. Calculation was performed in the 
same scheme as that of Fig. 6 but in the commensurate (Sic —> 0) limit [43| . 



wave vector Q = Qo + q will be investigated. The optimal q is numerically found by replacing Qo in eq. (145[) and 

(2) 

the ensuing expressions with Qo + q and minimizing fm with respect to q. Through our examination of possible 
q, we have found that a diagonal q consistent with that determined experimentally in CeCoIn5 [l6| is not easily 
obtained in the case where the nesting condition on the Fermi surface is relatively kept even if deviating from the 
diagonal (i.e., Qo) direction. The dispersion relation (|44p with vanishing t$ corresponds to this case. As shown in 
Fi g. 7. the Fermi surface with a remarkable inflection close to the diagonal direction is needed to obtain a diagonal 



16j . Here, we have assumed that the relevant Fermi surface to the d-wave superconductivity and the itinerant 



antiferromagnetism is the so-called 7-sheet [36j or the band-14 electron one [40j which has the largest density of 
states and is not cylindrical. As examined previously [26] , such a noncylindrical and 3D-like modulation of the Fermi 
surface is necessary to theoretically explain the presence of a longitudinal FFLO state in CeCoIn 5 in H || c suggested 
from a NMR experiment [3{|. On the other hand, it should be noted that, in contrast to the above-mentioned severe 
condition on the required modulation wave vector of the high field AFM order of CeCoIns, the AFM ordering itself 
close to H C 2(0) is realized as a consequence of the strong PPB and the c? a .2_j / 2-pairing symmetry irrespective of the 
details of the Fermi surface. 

In addition, we have also examined the field and temperature dependences of q. As Fig. 8 shows, the obtained 
diagonal q is mostly robust on sweeping the temperature and the magnetic field. This fact consistent with the 
experimental data [lfj} supports our picture that, in contrast to that in other work [4l|, the origin of the AFM order 
in the HFLT phase of CeCoIn 5 is not a FFLO structure but of a purely electronic origin |42j . 



H || c 

In obtaining Figs. 6, 7, and 8, we have used the approach in the Pauli limit (see sec. II). As mentioned earlier, this 
approach is not necessarily unrealistic in examining thermodynamic data of quasi 2D materials in H 1 c. In contrast, 
the corresponding neglect of the orbital pair-breaking in H | c is usually unacceptable because of the important roles 
of the vortices in this configuration. Nevertheless, it will be useful to know the PPB-induced AFM ordering in the 
Pauli limit in this configuration. In Fig. 9, we show an example of our results in H || c obtained in the Pauli limit. In 
contrast to Fig. 4, the field at which the AFM ordering is the most remarkable is much closer to H C 2(0), irrespective 
of the position of an AFM-QCP suggested from the data in the normal state (see Fig.l and the discussion relevant to 
Fig. 4 in sec. III). We believe that this difference between Fig. 4 and 10 is intrinsic and is a reflection of the difference 
in the magnitude of PPB. To clarify this issue further, one would need to perform a more elaborate analysis taking 
account of both the PPB and the orbital pair-breaking on an equal footing in future. 

In Fig. 9, the corresponding —Xs(H) curve in the case of (i^y-pairing has also been presented, for comparison, which 
implies that, even in H || c, the (icy-pairing case does not lead to the PPB-induced AFM ordering. This close relation 
between the diagonal AFM Q- vector and the direction of the gap node indicates that the four-fold symmetric d-wave 
SC symmetry in a SC material showing an AFM ordering enhanced on approaching H C 2(0) from below with increasing 
field should be always the d x 2_ y 2-one. 
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FIG. 10: Theoretical H-T phase diagrams in H 1 c following from our calculation in the Pauli limit. The symbols SC, FFLO, 
and AF (n) denote the uniform SC order (see the text), the FFLO one, and the AFM one defined in Fig.ll, respectively. The 
//^-transition on the dotted curve is discontinuous, while the FFLO and AFM orders disappears continuously on the red solid 
and blue solid curves, respectively. The used values of the parameters are ti = /J, = 20T C , fe/ii = —1.25, and tz/t\ — 0.35 in 
(a), while, in (b), they are the same as those in Fig.7. In the direction parallel to H, the AFM order in (a) has the in-phase 
structure relative to the FFLO modulation in any field range (see Fig.ll (2) below). However, this AFM order is lost (i.e., m 
vanishes) in the FFLO state with finite Qlo, and the incommensurate part, q, of the AFM wavevector Q is parallel to [1,0,0], 
in contrast to the observation [l5l. [Ir3|. On the other hand, in (b), the AFM order shows a structural transition [3| between 
the two structures shown in Fig.ll within the FFLO state, and, with decreasing H, the AFM order is continuously lost on the 
red solid curve in the manner accompanied by the dis app earance of the FFLO nodal planes. The corresponding q is shown in 
Fig. 8 and is parallel to [1,1,0] as seen in experiments [la, llq|. 



V. EFFECT OF FFLO MODULATION ON AFM ORDERING IN PAULI LIMIT 

In the preceding sections, we have focused on the case with a finite AFM transition temperature in some held 
range just below £f C 2(0). From such a case, the situation with a remarkable AFM critical fluctuation near -ff C 2(0) 
but with no genuine AFM order is easily created by reducing the repulsive interaction strength U. As mentioned 
in Introduction, however, when discussing the HFLT phase of CeCoIns in H _L c with n || c, it is indispensable to, 
consistently, take account of appearance there of a different ordered state from the AFM order induced by the strong 
PPB, such as the spatial modulation of the SC order parameter of the longitudinal FFLO state. 

Here, it is pointed out that the FFLO spatial modulation of the SC order parameter in H _L c significantly enhances 
the PPB-induced AFM ordering. In the context of CeCoIns, it is an elaborate task to give a full description of this 
interplay of the two orderings within the two approaches explained in sec. II. Here, we examine this interplay by 
assuming the wavelength of the FFLO modulation 2n/qLo, defined through eq. (|42[) . to be long enough to neglect 
the gradient terms in the free energy on the AFM ordering. Implication of this local approximation on the AFM 
modulation will be explained at the end of this section. 

Possible phase diagrams including the FFLO and AFM ordered states and resulting from our calculation in the 
Pauli limit are shown in Figs. 10 and 12. There, the uniform SC state in lower fields in the Pauli limit corresponds to 
the ordinary vortex lattice in the full description including the orbital pair-breaking effect. In the present formulation 
in the Pauli limit, a possible FFLO state is found by substituting the test solution eq. p^]) of the SC order parameter 
A into the free energy terms given by eq. (|41[) and minimizing them with respect to <?lo- Then, in our formulation in 
the Pauli limit, the i? c2 -transition was of first order, while the resulting transition between the FFLO and uniform 
SC states was of second order at any temperature, implying that the distance between the neighboring FFLO nodal 
planes diverges at the transition. Note that, in considering the FFLO ordering, the averaged value of |A| is so rigid 
that it may be assumed to be unaffected by an AFM ordering. 

Then, when considering a coexistence of the AFM and FFLO orderings in the local approximation for the AFM 
modulation, we only have to use eqs. ([50|) and (f5Tj) with A replaced with A(r) given in eq.([42|). We have checked that, 
in all cases we have examined, fm^ > so that the AFM transition is continuous by itself. First, it will be clarified how 
spatial variation of m should be realized in the LO structure, eq. fh?"]) . of A. It is easily understood by recalling the role 
of the coupling term f^' m in the perturbative approach that, when f& m > (< 0), the structure of the AFM order 
parameter showing the our-of-phase (in-phase) modulation for the SC order parameter is the most stable, and that the 
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spatially uniform AFM order is unstable in the FFLO state. It is directly concluded from this consideration that, in 
the presence of the Larkin-Ovchinnikov state, eq. (|42]) . the uniform AFM order cannot be stable, and that, at least in 
higher fields where | A| is smaller, the in-phase structure, Fig. 11 (2), is the most stable as a direct consequence of the 
PPB-induced AFM ordering. On the other hand, in lower fields and particularly close to the second order transition 
to the uniform SC state, we have a couple of candidates of possible phase diagrams, and it is not easy to predict 
which of them should be realized in a particular material. In fact, to clarify the best candidate for CeCoIns, extensive 
consideration is necessary as follows. First, an experimental fact [44j that the anomalous doping (impurity) effect on 
the second order transition in CeCoIn 5 [l|| [5(| , indicating the presence of the FFLO state above the transition 21 1 , is 
seen entirely over the field range occupied by the HFLT phase implies that CeCoIn 5 at ambient pressure does not show 
realization of the phase diagram of the type of Fig. 12 in which a direct transition between the AFM and the uniform 
SC phases occurs without the FFLO state at lower temperatures. On the other hand, it is not easy to theoretically 
justify a simultaneous disappearance of the in-phase AFM order, sketched in Fig. 11 (2), and the FFLO one at the 
same second order transition where 27t/<7lo diverges: Within this scenario, a simultaneous disappearance of the two 
orders would require a discontinuous vanishing of the AFM order parameter \m\ in contrast to the observations in 
CeCoIns. For this reason, we propose two candidates, following from our microscopic calculations, of the high field 
phase diagram of CeCoIri5 in Fig. 10. In (a), the PPB-induced in-phase AFM order, sketched in Fig. 11 (2), is lost 
within the FFLO state just above the second order transition, while the longitudinal modulation of the AFM order 
changes from the in-phase modulation to the out-of-phase one sketched in Fig. 11 (1) on approaching the second order 
transition, as a result of the reduction of PPB. The out-of-phase AFM modulation is consistent with the conventional 



picture |41|, |45| that a spatial region where |A| is small is occupied by a competitive non SC order. In this case, 
the AFM order rides on the nodal planes and thus, is continuously lost through the continuous disappearance of the 
FFLO nodal planes, while the PPB mechanism of the AFM order favors the local coexistence with the SC order, i.e., a 
structure like Fig. 11 (2). In any case, an additional continuous transition [46| inevitably appears within the resulting 
HFLT phase of the theoretical phase diagrams appropriate for CeCoIns . If taking account of the consistency on the 



direction of AFM-Q vector between the experiment [15|,[l6j and our results, Fig. 10 (b) with the diagonal q shown in 
Fig. 8, becomes the best candidate as the phase diagram of CeCoIns, while the electronic parameters (see the caption 
of Fig. 10) resulting in Fig. 10 (a) never leads to a q in the diagonal direction. However, this correspondence between 
the longitudinal structure of the AFM order and the detailed direction of the AFM Q-vector is quite a subtle issue 
and might be changed due to a further refinement of the starting electronic model. For this reason, it appears that 
we should not conclude here which of the two figures in Fig. 10 is more appropriate for CeCoIns. 

The presence of the wide FFLO phase with no AFM order at higher temperatures in Fig. 10 might indicate that the 
present picture on the HFLT phase of CeCoIns is insufficient, because such a nonmagnetic FFLO region has not been 
identified so far in experiments on this material. However, the present analysis in the Pauli limit where ap.i = oo 
certainly overestimated the temperature region of the FFLO phase. In an improved analysis for finite ap. ^-values 
to be performed in future works, the FFLO region unaccompanied by the AFM order is certainly expected to be 
narrower if taking account of the orbital pair-breaking within the present approach. 

Here, we comment on justification of our use of the local approximation on the AFM modulation. As far as the 
transition between the FFLO and uniform SC states is of second order, this local approximation is always justified 
at least just above the transition because the order parameter [1 71 ] of this transition, <7lOj is inversely proportional to 




FIG. 11: Two structures, (1) and (2), of the spatial modulation of the AFM order parameter m relative to that of A in the 
coupled FFLO and AFM phase. The AF(n) (n = 1 and 2) in Fig. 10 corresponds to the structure (n) in this figures. For 
simplicity, we have described the AFM order in the form sin(gLoa; + <j>o). The structure (1) ((2)) corresponds to the case with 
4>o = (<j)o = 7r/2). The phase (j>o continuously changes with increasing H in the interval between zero and n/2. 
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FIG. 12: Another theoretically possible phase diagram including the FFLO and AFM orders following from the parameter 
values, ti = 10T C , tz/ti = 0.02, and tz = fi = 0. The blue solid curves are the actual AFM transition lines, and, in the absence 
of the FFLO state, the dashed blue becomes the AFM transition line at higher fields. This case in which the AFM order 
appears without the FFLO state in lower fields does not apply to the high field phase diagram of CeCoIns (see the text for 
details) . 




the distance between the neighboring nodal planes which diverges at the transition. At higher fields, however, gLO 
grows so that the neglect of the gradient terms may not be justified in general. If so, the above results in the local 
approximation would be quantitatively changed by the gradient terms. In the case of CeCoIns, however, the presence 
of the strong PPB-induced AFM fluctuation induces [26| a quasiparticle damping which destabilizes the transverse 
FFLO states to be described by higher Landau level modes of the SC order parameter compared with the longitudinal 
FFLO state, eq. (l42l) . Further, this increase of the quasiparticle damping even weakens effects of PPB and results in 
a reduction of qlo- We expect the use of the local approximation for the AFM ordering to be justified in this sense. 

It should also be noted that the spatial variation due to the vortices in the real systems with the orbital pair- 
breaking has not been incorporated in this section. In fact, as in the AFM ordering detected in the vortex core of 



high T c cuprates 47|, it is natural to expect appearance of an AFM order in the spatial region where the SC order 



is weaker like the vortex core. Even in the present case with strong PPB, the spatial modulation perpendicular to 
H of the SC order parameter due to the vortex structure assists the AFM ordering, although its effect is found to 
be weaker than that of the FFLO modulation. On the other hand, contrary to the event seen in cuprates [13], the 
present PPB-induced AFM order tends to coexist with the SC order so that the AFM order parameter value |m| is 
maximal outside the vortex core. Details of this issue will be reported elsewhere |3lj |. 



VI. SUMMARY AND CONCLUDING REMARKS 

The phase diagrams shown in Fig. 10 are comparable with that following from a recent experiment (28j . In the 
NMR measurement in Ref. [28[ , the presence of both the AFM order and the normal state region has been detected in 
the HFLT phase of CeCoIns at 50 (mK) in H _L c, and the square-root field dependence of the quasiparticle number 
consistent with that of the FFLO order parameter, i.e., q^o ^ yH — H2 (T) , has been found, where H2(T) is the 
field at which the second order transition to the ordinary vortex lattice phase (the uniform SC phase in the present 
Fig. 10) occurs. Another crucial observation in Ref.f^ is that, in contrast to the picture in Ref. (4l| ■ the observed 
AFM order is extended spatially without being localized in a narrow spatial region. It appears that the in-phase 
structure in Fig. 11 (2), i.e., the coexistence of the AFM and SC orders induced by PPB, is the consistent picture with 
this experimental fact. 

In sec. IV, we have examined the direction of the modulation wave vector Q of the stable AFM order originating 
from the PPB-induced mechanism and have found that, including its incommensurate component q, Q can take the 
orientation parallel to the direction of the gap node irrespective of the H-direction in the a-b plane. To the best of 
our knowledge, this is the first study giving a consistent calculation result on q with the data [IH [l6| in the HFLT 
phase of CeCoIns. In relation to this, we note that we could not find a diagonal q- vector in terms of the more 
familiar tight-binding electronic Hamiltonian (48j | with no £3 term in the range of the values we have assumed for the 
parameters t\, £2, and fi. In any case, it should be stressed that this Q-direction is highly sensitive to the details of 
the starting electronic model. 
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Recent neutron scattering experiments [49( on CeCoIns in the field directions tilted from the a-b plane have shown 
that just 17 degrees' tilt of the applied field results in disappearance of the AFM order present in H 1 c. Note that 
the focus of this experiment is the AFM order and not the tilt direction signaling disappearance of the HFLT phase, 
i.e., of the FFLO order. It is interesting to note that, according to another observation resulting from the field 
tilt, the HFLT phase survives up to 20 degrees, suggestive of the presence of the FFLO state with no AFM order in 
a narrow high angle region. In relation to this, we point out that such an AFM order disappearing separately from 
the FFLO order due to the field tilt is not surprising from the view point of the present theory, because, as is seen by 
comparing Fig. 3 with Fig. 4, the PPB-induced AFM ordering in H |j c is much weaker than that in H 1 c as far as, 
as have been assumed throughout this paper, n is locked to the c-axis. A further theoretical study on this issue may 
be useful for confirming the genuine picture on the HFLT phase. 

In relation to this disappearance of the AFM order due to the field-tilt, a mechanism of AFM ordering of CeCoIns 



below H C 2 has been argued in Ref. [5l| where it results from the four- fold symmetric enhancement of the density of 
states in the vortex lattice. However, the Fermi surfaces assumed there [5l| to support the AFM order are the nearly 
cylindrical band 15-electron ones, in the notation of RefJ40|, with a smaller density of states, while the origin of the 
e?-wave superconductivity and the FFLO state in H || c [39| is the noncylindrical (3D- like) (2(| band 14-electron Fermi 
surface [40j corresponding to that of Fig. 7 (see the text of sec. IV). It is unreasonable for the Fermi surface relevant 
to superconductivity to change with tilting the magnetic field. While preparing the final version of our manuscript, 



we were aware of another proposal |52| on the AFM ordering in H 1 c below H C 2 in which attention is paid to the 
Zccman-splittcd nodal quasiparticles as the_origin of the AFM order. Judging from the similarity on the starting 
model, the contribution mentioned in Ref. [HI to the AFM ordering should be already included in the present theory 
with no limitation on the quasiparticles close to the gap nodes. However, we remark that focusing [52| on the Zeeman- 
splitted nodal quasiparticles would result in a remarkable field dependence of the incommensurate component q of 



the AFM wave vector Q, in contrast to our result in Fig. 8 consistent with the observation [16[. Further, we stress 



here that the argument in Ref.[21[, given in relation to Fig. 2 there, is also applicable to any picture 51, 52] identifying 
the experimental second order transition with a pure AFM transition and thus that the doping effect [Til [2(| leading 
to an extremely dramatic suppression of the transition with no notable change of the transition point to the HFLT 
phase is incompatible with such purely AFM scenarios on the second order transition (see also sec. I). 

In this manuscript, we have also stressed a close relation between the Q-vector of the AFM order or fluctuation 
enhanced close to H C 2(0) and the nodal direction of the ci-wave SC energy gap. Thus, if a novel SC material is 
accompanied by such an AFM ordering in high fields, it may become a useful method for obtaining information on 
the nodal direction of the SC pairing symmetry. We wish to stress that, in contrast to measurements of the thermal 
conductivity [22[ and the specific heat [53| in which experiments at very low temperatures have been necessary to 
determine the pairing symmetry, the present method does not require measurements at such low enough temperatures 
to obtain knowledge on the pairing symmetry. 
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APPENDIX 

(4) (2 4") 

Here, detailed expressions on fm and f& m ^° be use d in determining the character of the AFM transition in the 
perturbative approach will be listed. As far as the orbital pair- breaking effect is neglected in /^m > they are expressed 
by 
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Here, tp( n \z) is the n-th polygamma function satisfying 
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